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ABSTRACT 

The author presents critical comments on a paper by 
John Siegfried, in which Siegfried reports on a controlled experiment 
to determine whether the educational experience of being a proctor is 
sufficiently valuable to justify awarding academic credit. Siegfried 
argues that a semester of proctoring teaches a student more than a 
one-semester, upper-level economics course and that credit should be 
given. Tietenberg disagrees. When two groups of students, comprised 
of proctors and students in an upper-level course, were given the 
College Level Examination Program (CLEP) , the proctors scored 
highest. However, Tietenberg claims, the proctors had been exposed to 
an introductory course which the CLEP content focuses on, while 
students in upper-level courses dealt in-depth with specific areas of 
economics which the CLEP is not designed to measure. The educational 
outputs of the two types of courses are different, and it is 
difficult for a single instrument to adequately measure the outputs 
on a one- dimensional scale. In terms of costs and benefits of 
proctoring, Tietenberg notes that proctoring takes more of a 
student's time than taking an upper-level elective. His research 
shows self-pacing effects and workload effects from increased time 
requirements on Personalized System of Instruction (PSI) students. 
(AV) 
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Introduction 

As I see it the role of a discussant is to serve as a representative of the listening 
audience, one who has read the work in question carefully and who is prepared to assess 
what conclusions can validly be drawn from the study. This I will attempt to do by 
commenting critically on several aspects of the paper^ and then will give my version 
of what I think the paper does and does not say. 

Before launching into a critical review I would like to put my criticisms into 
perspective by acknowledging at the outset that I believe this to be a very useful 
addition to the literature on the use of student proctors or teaching assistants. Awarding 
academic credit rather than paying these proctors would be a very effective way to 
reduce the costs associated with teaching courses using these proctors. The question 
then is whether the educational experience associated with being a proctor is sufficiently 
"valuable" as to justify the awarding of academic credit. The Siegfried paper reports 
on a controlled experiment which, in principle, could answer the question. 

As I shall make clear below I believe the evidence in this paper, when properly 
interpreted, while a very useful start in the right direction, is not sufficiently 
convincing to resolve the issue. Let me make clear the nature of my reservations. 

The Dimensionality of a Value-Added Measure 

That proctoring is something which leads to a greater mastery of the material accords 
well with mo$t of our causal observations I suspect and I find the Siegfried test of that 
hypothesis thoroughly convincing. However, the essence of his argument is that a semester 
of proctoring teaches d student more than a one semester upper level economics course. 

bohn J . Siegfried, "Is Teaching the Best Way to Learn? An Evaluation of the Benefits 
and Costs to Undergraduate Student Proctors in Elementary Economics, " a paper 
delivered to the American Economic Association meetings in Atlantic City, New Jersey, 
September 17, 1976. 



This is the basis For his conclusion that academic credit should be given and it is a much 
stronger hypothesis. The support For this stronger hypothesis I Find weak. 

The source oF my reservations is the test instrument used to measure value added For 
both the proctoring experience and the upper level course. This exam, th&CLEP is 
designed to test the student's basic understanding oF economic concepts and principles. 
The building oF such an understanding is a chieF purpose oF the introductory course and 
thereFore it is not surprising that the proctors do well on the CLEF There is a basic 
compatibility between the content oF the course and the questions cn the CLE? 

This kind oF compatibility does not exist between the normal upper level elective 
and the CLEF . At Williams, For example, the upper level electivesare specialized 
courses dealing intensively with a particular Field within economics. While I did not have 
access to the Vanderbilt catalog while I was writing this, I rather suspect the some is true 
there as well . Thus they are not intended to sharpen general knowledge about theory, but 
rather to provide greater depth and experience in a particular area. As I think about some 
oFour upper level courses — econometrics, urban economics, law and economics — I 
would not expect a CLEF type oF exam to be a valid instrument For testing what is learned 
in those pourses. 

To make my point as graphically as possible had SiegFried given everyone an econometrics 
exam I am sure that he would have derived the conclusion that proctoring was not nearly 
as valuable as taking the econometrics course. My conclusion is that while the proctoring 
experience is clearly educationally valuable it is misleading to conclude, as SiegFried 
does, t*hat it is more valuable than taking an upper level course in economics. It is correct 
to say tha) more basic concepts and principles are learned by proctoring an introductory 
course than by taking an upper level course; it is not correct to say that more is learned. 
The educational outputs oF the two types oF courses are simply diFFerent and it is diFFicult to 
think oF a single test instrument which would adequately measure these outputs on a 
unidimensional scale. The CLEF ;$ not such an exam. 
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The Treatment *emal Costs 

The second n criticism I have of the paper, which is equally applicable to most 
of the literature in the Field, is the partial equilibrium nature of it. The discussion of 
costs and benefits is conducted as if economics were the only subject taken. This criticism 
IS most applicable to Siegfried^ s discussion of the large amounts of time consumed by 
proctoring. Proctoring takes more time than taking an upper level elective. The question 
is where does this time come from - from the student's other courses? Are we creating an 
externality on the rest of the proctor's courses by using undergraduate tutors? I recognize 
that this question is more appropriate to deciding whether to have tutors or not rather than 
whether to give them academic credit, but I regard this logically prior question as still 
unresolved. 

2 3 * • . 

In my own published v>;rk on comparing the external effects of the increased time 

requirements on PSI students (not proctors) I identified two possible effects. The first, the 

seif^acing effect, allows the student to allocate this time flexibly during the semester. 

This should have a positive effect on other courses since the student can allocate time to 

those courses when time is most important. The second effect, the workload effect, reflects 

the fact that a PSI course simply takes more total time during thc^ se^mester. To the extent 

that this time comes directly from study time for other courses they would suffer. The effect 

on other courses then is an empirical question to be resolved through an appeal to the data. 

My results were that PSI students did somewhat better in their other courses than the control 

group though the difference was not statistically significant, I concluded that for the 

students the external costs appear to be zero. 



T. H. Tietenberg, "Teaching Intermediate Microeconomics Using the Personalized System 
of Instruction, " Behavior Research and. Technology in Higher Education (Springfield: Charles 
C. Thomas, Publisher, 1975), 75-8y. A somewhat longer unpubiished paper , "A Multi- 
dimensional Comparitive Analysis of the Personalized System of Instruction with the Lecture 
Method" containing more data and more uses of it is available from the author on request. 

^he effects in question are external in the sense that they don't affect performance in economics 
but they are internal to the proctor. 



In this context, however, it seems to me that the proctors, in contrast to the students, 
do not have the self-pacing option. The demands on their time are not self-regulated 
Thus it is an open question whether there are costs imposed on the rest of the student's 
educational experience when employed as a proctor. I would hope that John and others 
would explore this question. My own sample sizes are so small as to prohibit me from cominp 
to any conclusions. 



Note that my view that the burden of the time requirement is greater when the student 
tonnot control its allocation is in part corroborated by the interview evidence introduced 
by Siegfried. This evidence suggests that when proctor availability requirements were 
regularized by fixed office hours proctor complaints went down. 

Er|c 6 



